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(54) A digital interactive system for providing full interactivity with live programming events 



(57) The present invention relates to an interactive 
digital system (1) enabling viewers full and active partic- 
ipation in experiencing a live broadcast event. Particu- 
larly, the presentation of the live event is personalized 
for the viewer through the provision of options, including 
multiple video streams, associated with different cam- 
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era angles, for example, and integrated audio and 
graphic segments. Further, information obtained from 
related Web sites can be integrated into the live pro- 
gram. 
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Description 

[0001] Interactive video and audio presentation 
systems are currently being introduced into the enter- 
tainment and educational industries. A prominent inter- 5 
active technology that has been applied successfully in 
these industries is based on providing interactivity in a 
one-way system through the provision of multiple time- 
synched parallel channels of information. For example, 
U.S. patents nos. 4,264,925 and 4,264,924, provide 
both audio and video interactivity and disclose interac- 
tive television systems where switching among multiple 
broadcast or cable channels based on viewer selections 
provides an interactive capability. 
[0002] These systems have been enhanced to 
include memory functions using computer logic and 
memory, where selection of system responses played to 
the viewer is based on the processing and storage of 
subscriber responses, as described in U.S. patent no. 
4,507,680. 

[0003] The benefits of providing interactivity 
through the use of different audio responses is dis- 
closed in Freeman, U.S. patent nos. 4,847,698, 
4,847,699 and 4,847,700. These television systems 
provide a common video signal accompanied by several 
synchronized audio channels to provide content related 
user selectable responses. The audio signals produce 
different audio responses, and in some cases, these are 
syllable synched to a first audio script and to the video 
signal (such as to a person or character on a display), 
providing the perception that the person's or character's 
mouth movements match the spoken words. 
[0004] Interactivity is brought to the classroom in 
the Freeman U.S. patent no. 5,537,141. The distance 
learning system claimed in this application enhances 
the classroom educational experience through an inno- 
vative use of interactive technology over transmission 
independent media. When an instructor, either broad- 
cast live on video or displayed from videotape, asks a 
question, each and every student responds, preferably 
by entering a response on a remote handset, and each 
student immediately receives a distinct and substantive 
audio response to his or her unique selection. The indi- 
vidualization of audio response from the interactive pro- 
gram is a major aspect of the invention. 
[0005] Individualization of audio is brought to the 
home based on the technology disclosed in Freeman 
U.S. patent no. 5,585,858. This system provides a pro- 
gram that can be watched on any conventional televi- 
sion set or multimedia computer as a normal program. 
But if the viewer has a special interactive program box 
connected to the television, he or she can experience a 
fully functional interactive program. Each interactive 
viewer enjoys personalized audio responses and video 
graphics overlayed on the screen. The interactive pro- 
gram can be provided to television sets or to computers 
by cable, direct broadcast satellite, television broadcast 
or other transmission means, and can be analog or dig- 



ital. 

[0006] However, what is needed is an interactive 
presentation system for providing true video, audio and 
graphics interactivity with digital programs broadcast 
live. Such a system must efficiently package all the dig- 
ital elements of the live interactive program at a central- 
ized control studio and allow viewers at home to receive 
personalized interactive programming. 

SUMMARY OF THE INVENTION 

[0007] In accordance with the invention, there is 
provided an interactive digital system allowing the 
viewer active participation in selecting digital video 
streams, associated with different camera angles, for 
example, and integrated audio and/or graphics seg- 
ments. Further, Web pages from Internet Web sites can 
be integrated into the program. The invention is particu- 
larly suited for the environment of live events, such as 
the broadcast of live sporting events. The viewer can 
appear to direct the camera shots by instantly changing 
among various camera angles, choose player inter- 
views, or display associated statistical data on the team 
or players via graphics. In this manner, the system 
allows the individual subscriber to act as if he or she has 
control over how the program is directed and presented 
on their personal television set. In addition to selecting 
different camera angles, various audio options 1 close- 
ups, slow motion, replays, graphics overlays, graphics 
or audio from Web sites, etc., are all possible. Further, 
games can be integrated with the live sports program- 
ming to increase viewer interest. 
[0008] Thus, viewers can customize the content of 
programs. The interactive digital programming of the 
present invention is particularly advantageous for view- 
ing live sporting events. Viewers are not limited to 
selecting from multiple camera angles, but may also call 
up player statistics on demand, listen to selected player 
interviews, etc. Cameras can be focused on different 
segments of an event. Further, video options could 
include video replay, slow motion effects, isolation on a 
particular player or group, etc. Changes are seamless, 
thereby adding to the effect that the viewer is directing 
the television show just as a director now does from a 
control room. 

[0009] This "director" role by the viewer is possible 
due to the interactive technology of the present inven- 
tion and also due to the digital compression and trans- 
mission scheme which allows for much greater 
information throughput over a given bandwidth, allowing 
viewers to choose from angles that are already availa- 
ble but presently cut by the director. 
[0010] The digital interactive system is based upon 
branches which occur in the course of the full-motion 
video. Branches are real-time parallel paths that may be 
other full-motion video segments, graphics which are 
integrated into the video, audio segments, and/or 
retrieved Web pages which are integrated into the live 
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event. 

[001 1 ] Sometimes, the interactive digital system will 
act upon the viewer's response immediately; other 
times, it will utilize ACTV's unique "profiling" concept to 
act upon the response later. This technology enables 
the system to "remember" the viewer's responses and 
desires, and integrate them into the video, audio, graph- 
ics and/or Web site information at a later point. For 
example, the viewer could specify at the beginning of a 
football game to isolate the offensive quarterback of a 
particular team. Thus, whenever the team of choice is 
on offense, the video isolation of the quarterback is dis- 
played to the viewer automatically. Or, based on how a 
viewer has selected camera angles, replays, etc., over 
the past five minutes, the system acts to mimic these 
selections at later times during the program. The sys- 
tem of the present invention "learns" from the viewer 
how they want to view the game, and thus, continues 
viewer selection sequences made earlier. 
[0012] At the source, the present invention com- 
prises a plurality of video cameras, each of the video 
cameras relaying a different predetermined view of an 
event. The video signals corresponding to the different 
cameras are forwarded to a central control studio. Fur- 
ther, one or more audio signals or graphic statistical 
overlays can be collected and sent to a central control 
studio. After receiving the video, audio, and graphics 
signals at the central control studio, these signals are 
digitized and compressed in digital video and audio 
compressors. These signals are then combined with 
special data codes into a "digital package," and subse- 
quently, transmitted over a cable distribution system. 
The special data codes are the keys to unlocking the 
interactive potential of the program. 
[0013] The digital program signals are transmitted 
to a receive site by any suitable transmission means. 
Once received by a receive antenna, the digital program 
signals are passed along on a digital cable television 
distribution system to the viewer homes. Further, some 
other signals or commercials can be inserted at the 
local head end. The signals are received and processed 
in a digital cable box. Selections of the video, audio, 
graphics display and/or Web pages can be made as a 
function of immediate viewer entries, or to interrogatory 
responses presented at the beginning or during the pro- 
gram, or based on a prestored viewer profile. Once a 
decision is made to switch from one video option to 
another video option, the digital switch is performed 
seamlessly. 

[0014] As mentioned above, the program at prede- 
termined times or immediately upon user entry can 
retrieve and branch to informative segments from Web 
sites. For example, a viewer watching a sporting event, 
through the system of the present invention, can receive 
a stream of Web pages which provide additional, spe- 
cific information relating to a favorite player, team or per- 
haps the remaining schedule for the sports team, as 
examples. In addition, users can take advantage of the 



two-way capabilities of the Internet to respond to polls 
or to link to additional sites. 

[0015] Another Internet-based application allows 
advertisers to speak more directly to consumers by 

5 directly sending Web pages to the consumer instead of 
merely displaying Web addresses in their commercials. 
The particular advertising information from Web sites 
can be targeted to viewers based on the viewer profile, 
stored either in the digital set top box or at the cable 

10 headend. Alternatively, Web site access can be initiated 
by the viewer by simply clicking on the remote during 
the commercial. Thus, viewers have the capability to 
individually select Web sites if they want more informa- 
tion from advertisers, for example. 

15 [0016] The video programming and corresponding 
Internet pages can be viewed either on personal com- 
puters equipped with a television card on special digital 
cable boxes with stored interactive Internet application 
software providing Internet access, or on digital televi- 

20 sion sets, ail of which would utilize the specialized 
TV/Internet software of the present invention. 
[0017] The present invention also has applications 
for other types of programming. For example, viewers 
can direct the scenes of a murder mystery. Switching 

25 from one scene to another can be done seamlessly 
without noticeable effect on the viewer. Further, the 
present invention can be used for any kind of live or pre- 
recorded event. For instance, a music concert or a polit- 
ical convention can be enhanced in the manner of the 

30 invention. 

[0018] Accordingly, a primary objective of this 
invention is providing an enhanced digital live program 
allowing the display to be tailored to the user's desires, 
choices or interests. 

35 [001 9] It is an object of this invention to personalize 
and enhance live sporting events for the viewer. 

DESCRIPTION OF THE DRAWINGS 

40 [0020] 

Figure 1 is a diagram of the network and equipment 
for providing live digital programming. 

45 Figure 2 is a block diagram of an interactive digital 
cable box allowing seamless switching between 
video signals. 

Figure 3 is a block diagram of an alternative dual- 
50 tuner interactive digital cable box allowing seamless 
switching between video signals. 

Figure 4 is a block diagram of another alternative 
interactive digital cable box allowing seamless 
55 switching between video signals. 

Figure 5 is a time diagram showing a representa- 
tion of trigger points and corresponding video, 
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audio and / or graphics segments, one or a combi- 
nation of which are selected for presentation to the 
subscriber immediately after the execution of the 
trigger point function. 

5 

Figure 6 is a block diagram of an alternative embod- 
iment of the interactive system including Internet 
access. 

Figure 7 is a block diagram of the two-way configu- 10 
ration of the system. 

DESCRIPTION OF THE PREFERRED EMBODI- 
MENTS 

15 

[0021 ] The present invention is an interactive digital 
system 1 for producing a powerful personalized pro- 
gram allowing the home viewer an expanded set of pro- 
gramming options. Digital TV streams are put into digital 
packages made up of video, audio, data codes and 20 
graphics, and are used to provide personalized 
responses to viewer selections. Such responses can be 
further enhanced by allowing access to Internet Web 
sites 1 70. In this manner, sports such as golf, football, 
baseball, basketball, etc. can now be watched with 25 
greater interest and involvement. However, even further 
enhanced interactivity is possible with the present 
invention due to the provision of various profiling and 
memory features. 

[0022] As shown in Figure 1 , the present invention 30 
begins with the gathering of several possible video 
streams by way of cameras 100, strategically located at 
a sporting event 10, for example. Currently, many cam- 
eras 100 are employed at a sporting event 10. Super 
Bowl coverage, for example, typically encompasses 25 35 
to 30 cameras. These live video streams can be inte- 
grated with recorded video streams which, for example, 
could include highlights from the current game or past 
games, player profiles, etc. To describe the components 
and operation of the present invention, the production 40 
and transmission of a live sporting event 10, i.e., foot- 
ball, is chosen to present the invention features. How- 
ever, other applications can be appreciated by the 
reader, including several disclosed below. 

45 

A. The Control Studio 

[0023] The interactive broadcast program is pre- 
pared at the control studio 5 into digital packages. The 
control studio 5, as shown in Figure 1 , allows a producer 50 
to create and introduce interactive elements during a 
live broadcast. In one preferred embodiment, the pro- 
ducer pre-records a set of interrogatories or instructions 
for the user. These interrogatories may include such 
questions as the following: 55 

SELECT THE CAMERA ANGLES/OPTIONS THAT 
YOU PREFER: 



OPTIONS: END-ZONE 

FIFTY YARD LINE 
FOCUS ON THE QUARTERBACK 
FOCUS ON THE DEFENSE 
FOCUS ON THE CHEERLEADERS 
PROVIDE HIGHLIGHTS 

WHO IS YOUR FAVORITE TEAM IN THE GAME? 

OPTIONS: BLUE TEAM 
RED TEAM 

WHO IS YOUR FAVORITE PLAYER IN THE 
GAME? 

OPTIONS: RED RUNNER 

BLUE QUARTERBACK 
BLUE DEFENSIVE END 
RED QUARTERBACK 
ETC. 

ARE YOU INTERESTED IN RECEIVING STATIS- 
TICAL SUMMARIES? 

WOULD YOU ENJOY LISTENING TO PLAYER 
INTERVIEWS DURING THE GAME? 

[0024] Such interrogatories can be presented to the 
viewer at the beginning of the broadcast or scattered 
throughout the program. Interactive responses to such 
interrogatories would include video, audio and graphics 
personalized to the particular viewer. The preparation of 
the graphics for presenting such interrogatories occurs 
off-line at the control studio 5 using chyron or any graph- 
ics language. These interrogatories or instructions will 
ultimately be displayed to the home viewer, preferably in 
the form of graphics, to facilitate the interactive 
responses. The producer creates these graphic video 
slides of questions on a computer using the text editor 
and chyron. Associated with each question, the pro- 
ducer enters a number of possible viewer options. Then, 
the producer relates each possible user entry to one or 
more corresponding interactive responses. If the 
response is information from an Internet Web page, the 
producer will indicate the particular Uniform Resource 
Locator (URL) of the Web page. The producer sets a 
time stamp for when each particular question will 
appear during the program. As explained below in more 
detail, the viewer response to a query will be used to 
direct which video (and/or audio, graphics, or Web 
page) option will be provided to the viewer. In the 
present invention, the interactive response to the query 
can occur immediately following the entry of the viewer 
entry or at some predetermined later time in the pro- 
gram using "trigger points," 500 as explained in detail 
below. 

[0025] With respect to the video segments, cam- 
eras 100 are preferably trained on different segments of 
the sporting event 10. As is common with broadcasts of 
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a football game, for example, cameras 100 could be 
located in the endzone, press box, the field and at vari- 
ous other locations throughout the stadium. Further, 
various video options can be created including video 
replay, slow motion, isolation on cheerleaders, particu- 
lar player or group of players. Instant replays are cre- 
ated by delaying the live feed for a certain number of 
seconds. These video streams are sent to a control stu- 
dio 5. The control studio 5 contains the necessary 
equipment for packaging the program for delivery to the 
viewers. The studio 5 contains a video switcher 105 
which receives the live signals from the cameras 1 00 by 
way of various input lines. Further, lines carrying 
recorded video streams from one or more VCRs 110, 
computers or CD players feed into the video switcher 
105. The video switcher 105 also receives video inputs 
from the control computer 135. Further, various graph- 
ics screens, depicting, for example, sports team or 
player statistics can be designed with the control per- 
sonal computer 135 and forwarded to the digital video 
switch 105. The producer, via the control PC 135, 
directs which video options to pass through the video 
switcher 105. At the output of the video switcher 105, 
each of the different output video streams access a sep- 
arate encoder 125 and are all GENLOCKED, so that 
each video stream is synchronized with the other video 
streams. 

[0026] After encoding, the video streams are input 
into a video compressor 125. Preferably, the digital com- 
pression scheme is MPEG-2. Preferably, 64 Quadrature 
Amplitude Modulation (QAM) is used as the modulation 
scheme. In this digital embodiment, four channels of 
digitally-compressed video content would carry about 
27Mbps using 6 MHz of bandwidth. Alternatively, if 256 
QAM is employed, seven-to-one video compression can 
be achieved with the MPEG-2 scheme. While MPEG-2 
is the preferred compression scheme, the signals can 
be compressed according to any known standard 
including MPEG-1, JPEG, or other DCT coding 
scheme, wavelets, fractals or other transform or wave- 
form based technique. 

[0027] The control studio 5 also contains an audio 
switcher 115 which receives live audio signals from 
microphones or recorded audio from tape players 120, 
CDs, VCRs 110, etc.. The control computer 135 sends 
commands to the audio switcher 115 directing which 
audio options should pass through the switch 115. Fur- 
ther, in the audio switcher 1 15 the various audio signals 
can be aligned to match the various video signals in 
time. In addition, VCR audio output is received by the 
audio switcher 115. The present invention can accom- 
modate any number of audio signals as output from the 
audio switcher 115, as directed by the producer. The 
audio outputs are received by an digital audio 
encoder/compressor 130. The audio signals are then 
preferably sampled, encoded and compressed in the 
digital audio encoder/compressor 130. The encoding 
technique can be a waveform coding technique such as 



PCM, ADPCM or DM. Alternatively, the signals can be 
encoded using synthesizer or vocoder techniques such 
as MUSICAM, Linear Predictive Coding (LPC) r Adap- 
tive Predictive Coding (APC), and Sub-band coding. 
5 Generally, the transmission rate is about 256 kbps per 
audio for the stereo pair. 

[0028] The timing and control for integrating the var- 
ious multimedia elements is provided by the ACTV 
authoring language, a unique set of interactive data 

w codes to facilitate the interactive process. The data 
codes are stored in memory in the control computer 135 
as part of the ACTV programming language. The codes 
comprise commands, or branch codes, for branching 
between interactive options, timing signals for control- 

15 ling the interactive program, data or text, commands for 
termination and initiation or interactive program viewing, 
or triggers for executing macros. Preferably, these com- 
mands are output from the control computer 135 and 
multiplexed with the video streams in the MPEG-2 com- 

20 pressor 125, as shown in Figure 1. Interactive options 
that can be branched to based on the branch codes 
include video segments, audio segments, graphics seg- 
ments and/or identified Web pages. 
[0029] There are several commands in the ACTV 

25 authoring language that function to provide the interac- 
tive sports programming applications. These com- 
mands are explained below in Section D, which details 
programming applications. 

30 B. The Transmission System 

[0030] As shown in figure 1 , the digital interactive 
system 1 uses an interactive program delivery system 
with any transmission means including satellite 15, 

35 cable 1 50, wire or television broadcast 1 75 to deliver the 
interactive program (hereinafter "composite interactive 
program") from the control studio 5 for distribution to 
subscribers in their homes. At the control studio 5, the 
signals from the digital multiplexer 140 are convened to 

40 RF and distributed to a microwave 175, cable 150 or 
satellite 15 network. Preferably, the digital interactive 
signal is forwarded from the control studio 5 to a cable 
headend 150, and subsequently, sent to the homes via 
the cable network. 

45 [0031 ] The program is preferably the broadcast of a 
live event. For example, live sporting events with added 
interactive elements can be broadcast from the control 
studio 5. Such live interactive elements could be differ- 
ent camera angles 100, slow motion video, etc., as dis- 

50 cussed above, while also incorporating prerecorded 
interactive segments such as highlights. Alternatively, 
the program can be produced off-line and stored in a 
program storage means at the control studio 5. 
[0032] In a satellite broadcast transmission, the dig- 

55 ital interactive signals are transmitted to uplink equip- 
ment where they may be multiplexed, upconverted. 
modulated, amplified and transmitted by satellite 15 to 
the receiver site 155 for distribution to the homes. 
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[0033] At the reception end, the composite digital 
interactive signals enter a receiver 155 where the sig- 
nals are demultiplexed, downconverted, demodulated 
and then passed to a cable distribution system that 
directs the signals to the homes. Although a cable distri- 5 
bution system 150 is the preferred transmission media 
to the homes, the digital signals may also be distributed 
by any conventionally known technique including satel- 
lite 15 to digital satellite receivers 155 at the home, 
fiberoptics, low or high power broadcast television 1 75, w 
telephone lines, cellular networks, and similar technol- 
ogy can be used interchangeably with this program 
delivery system. 

C. The Interactive Digital Bo x 

[0034] The interactive digital box 25 is shown sche- 
matically in Figure 2. Preferably, the interactive digital 
box is a specially adapted digital cable box 25. The con- 
troller 260 determines what video, audio, graphics 
and/or Web pages to display based upon the interactive 
commands which it receives. Based upon the com- 
mands, it plays the appropriate video, audio, graphics or 
Web page options. The graphics can either be created 
and sent from the control studio 5 or the graphical 
images can be created at the interactive digital box 25 
based on instructions preferably in the interactive com- 
mands. The interactive digital box 25 connects to a tele- 
vision 165 or other display monitor. Further, the 
interactive digital box 25 can be connected to a digital 
television 195, in which case an RF modulator 245 is 
not necessary. Each downstream transmission reaches 
the subscriber's house, shown in Figure 2, preferably 
through a tap and drop cable. 
[0035] The user interacts with the program through 
the input device 20. Preferably, the input device 20 is a 
typical television remote. The user interface 270 may be 
an infrared, wireless, or wired receiver that receives 
information from the input device 20. 
[0036] Regardless of the type of input device 20, 
user inputs can be utilized by the present invention 
immediately, or at a later time, to result in personalized 
graphics, video and/or audio presentation. For example, 
the present invention utilizes "trigger points," 500 as 
described below, to enable subsequent branches 
among multimedia segments during the show. 
[0037] Certain commands are sent from the control 
studio 5 as part of the digital interactive programming to 
facilitate the collection of user entries. These com- 
mands are extracted at the digital demultiplexer 21 0 and 
sent to the controller 260 which performs the appropri- 
ate action based on the commands. Some of these 
commands are explained below. 

a. BEGIN INPUT EXTENDED 55 

The Begin Input command starts an input 
period during which the user may press one or 
more buttons to select his or her choice(s). The 



entry format of this command is set forth as follows: 

BEGIN INPUT {{N} {VALID KEYS} {DIFFER- 
ENT} {FEEDBACK} 

where: 

N The maximum number of keys 

that can be pressed. 
KEYS The valid keys that can be 

pressed. 

DIFFERENT Requires each key pressed to be 
different. 

FEEDBACK The feedback type provided to 
the viewer. 



d. MAP 

The Map command is used to map video or 
audio tracks to keys, for use in connection with the 
Begin Audio Choice and Begin Video Choice com- 
mands. If this command is omitted, Key 1 will map 
to Track or Channel 1 , Key 2 to Track or Channel 2 
etc. This command allows mapping any key to any 
channel. 



15 

b. BEGIN VIDEO CHOICE EXTENDED 

The Begin Video Choice begins an input period 
for disco mode. During disco mode, the video 
switches dynamically each time the viewer makes a 
20 selection. 

BEGIN_VIDEO_CHOICE MODE [KEYS] 
{FEEDBACK [AUDIO]} 

The disco mode allows the viewer to 
25 change channels at will, while the OneShot 

mode allows only one change of channel. 

MODE DISCO/ONESHOT 
KEYS The valid keys that can be 
30 pressed. 

FEEDBACK The feedback supplied to the user 
for the key(s) which are pressed. 

The mode Disco allows the viewer to 
35 change channels at will, while the OneShot 

mode allows only one change of channel. 

c. BEGIN AUDIO CHOICE EXTENDED 

The Begin Audio Choice begins an input period 
40 for disco mode. 

BEGIN_AUDIO_CHOICE MODE [KEYS] 
{FEEDBACK [AUDIO]} 

45 MODE DISCO/ONESHOT 

KEYS The valid keys that can be 
pressed. 

FEEDBACK The feedback supplied to the user 
for the key(s) which are pressed. 

50 
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MAP KEYS [TRACKS [T1,...TN/T1-TN] 
CHANNELS [C1,...CN/C1-CN]] 

The map statement maps audio tracks or video 
channels to keys, to enable audio or video choice s 
commands to effect changes to tracks other than 
the default tracks, which are that key 1 maps to 
track 1, key 2 to track 2, etc.. All the choice state- 
ments after a map statement will cause the tracks, 
video tracks or channels to be changed to those 10 
specified in the map statement. The acceptable 
numbers for the map command are from 1 to 8, for 
keys, video channels and audio channels. 

[0038] Other commands include those which allow 15 
for the following applications: (1 ) viewer profiling, to ena- 
ble the set top box 25 to "remember" viewer prefer- 
ences; (2) uploading viewer responses to a central 
location; (3) downloading of text and graphics, for dis- 
play using the graphics chip of the set top box 25; (4) the 20 
ability of the viewer to prepare his own video, based 
upon his selections of camera shots 100 and audio, 
which can be stored and replayed for the viewer. 
[0039] The interactive digital box 25 of the present 
invention enables seamless flicker-free transparent 25 
switching between the digital video signals. "Seamless" 
means that the switch from one video signal to another 
is user imperceptible. Because the video signals are 
running off the same clock, the interactive digital box 25 
is capable of providing a seamless digital switch from 30 
one video signal to another signal. The program dock 
reference necessary for the box to make this seamless 
switch is preferably embedded in the signal header. 
[0040] As shown in Figure 2, a CPU 260 is con- 
nected to an RF demodulator 200 and digital demulti- 35 
plexer 210. The CPU 260 directs demodulation and 
demultiplexing of the proper channel and data stream to 
obtain the correct video signal. Seamless switching can 
occur with MPEG-2 compressed signals since there are 
points within the frame wherein seamless switching can 40 
occur. Preferably, switches occur at an "I" frame, assum- 
ing the use of MPEG-2 compression. The selected 
video signal is determined either by examination of the 
user's input from user interface 270 and/or any other 
information or criteria (such as personal profile informa- 45 
tion) stored in RAM/ROM 265. For example, the 
RAM/ROM 265 could store commands provided within 
the video signals as discussed in U.S. patent no. 
4,602,279, and incorporated herein by reference. 
[0041] The RF demodulator 200 demodulates data so 
from the broadcast channel directed by the controller 
260. After the data stream is demodulated, it passes 
through a forward error correction circuit 205 into a dig- 
ital demultiplexer 210. The demultiplexer 210 is control- 
led by the controller 260 to provide a specific video 55 
signal out of a number of video signals which may be 
located within the data stream on the demodulated 
broadcast channel. The demultiplexed video signal is 



then decompressed and decoded by decompres- 
sor/decoder 215. The video signal is synchronized by a 
sync add circuit 220 and a sync generator 225. The 
video signal is then buffered by a video frame buffer 
230. The buffered video signal is modulated by a modu- 
lator 245 into a NTSC compatible signal. Such a modu- 
lator is not necessary if the selected signal is sent to a 
digital television 1 95. 

[0042] By using a video frame buffer 230 and delay- 
ing the viewing of a given signal, enough time is allowed 
for the decompressor/decoder 215 to lock onto, decom- 
press, convert to analog, and wait for the resultant verti- 
cal interval of a second video signal. For example, 
assume video signal A is currently being processed and 
transferred through the circuit shown in Figure 2 and 
displayed. Based upon a user selection, the controller 
260 directs the digital demultiplexer 21 0 and RF demod- 
ulator 200 to switch to another video signal, video signal 
B. To accomplish this, the analog video from the first 
digital video signal, video signal A, complete with video 
sync, is fed into video frame buffer 230. This buffer 230 
can hold the full video picture for "n M number of frames 
after which the signal is output to the display.. In effect, a 
delayed video signal A is viewed "n" number of frames 
after the signal has been received. When the user 
selects a different video path by means of pressing a 
button on a keypad or entry by other means, the control- 
ler 260 instructs the digital demultiplexer 210 to stop 
decoding signal A and lock onto signal B to begin 
decoding signal B instead of signal A. 
[0043] While this is happening, even though the 
decompressor/decoder 215 is no longer decompressing 
video signal A, the display is still showing video signal A 
because it is being read from the buffer 230. As soon as 
decompressing and decoding occurs, the controller 260 
looks for the next vertical blanking interval (VBI) and 
instructs the video frame buffer 230 to switch to its input, 
rather than its buffered output at the occurrence of the 
VBI. 

[0044] Since the RF demodulator 200, forward error 
corrector 205, digital demultiplexer 210, and decom- 
pressor/decoder 215 require a certain time period to 
decompress and decode the video signal B frame from 
its data stream, the size of the buffer 230 has to be large 
enough so that this processing can take place without 
interruption during the switching of the video signals. If 
desired, the system may continue to use the buffer in 
anticipation of a future switch. By using the controller 
260 to manipulate the fill and empty rate of the buffer 
230, the buffer 230 may be rapidly filled with video sig- 
nal B frames and then after a period of time will be reset 
and ready to make another switch to another video in 
the same manner. The buffer 230 may also be reset by 
skipping frames or providing a delay between sequen- 
tial frame outputs for a short time in order to fill the buffer 
230. If a delay is used to maintain video signal or frame 
output while the buffer 230 is being filled, a slight distor- 
tion may occur for a brief amount of time. 
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[0045] Because a first video signal is always dis- 
played as the output of the buffer 230 after the delay, the 
buffered video masks the acquisition and decoding of a 
second video signal. As long as the buffer 230 is large 
enough to keep the first video running while the second 
video is being decompressed and decoded, a seamless 
switch will occur. 

[0046] While the digital interactive box 25 of Figure 
2 provides video interactivity, audio and/or graphics 
interactivity is also provided. For example, if, based on 
the viewer profile or viewer response to query, it is 
determined that the viewer's primary language is Span- 
ish, then that viewer could obtain Spanish commentary 
to the football, soccer, etc. game. Alternatively, if a 
viewer has a favorite athlete, the audio can switch to an 
interview with the athlete during a segment of the 
broadcast. Multiple digital audio options forming a set of 
suitable responses to an interrogatory message can be 
sent as part of the composite digital signal. As set forth 
in U.S. patent no. 5,585,858, herein incorporated by ref- 
erence, there are a number of different ways to effec- 
tively forward the necessary audio options for a given 
live event to the digital interactive box 25. With the 
present invention, it makes no difference how the audio 
options reach the digital interactive box 25, as long as 
they are available for selection and play at the appropri- 
ate times. 

[0047] In Figure 2, the digital demultiplexer 210 
extracts the digital audio signal(s) and forwards them to 
the audio switch 250. Additional audio options are avail- 
able from the digital audio memory 255. At certain times 
during the program, the data codes will identify the 
selection of a particular audio option corresponding to 
previous user inputs. The controller 260 calls the appro- 
priate audio options from internal memory 255 or 
directs the audio switch 250 to select a predetermined 
audio segment received as part of the received digital 
signal for passage to the RF modulator 245 for play to 
the subscriber. At the end of the audio segment time 
period as indicated by the data codes, the controller 260 
instructs the audio switch 250 to again pick up standard 
audio. 

[0048] The digital demultiplexer 210 sends the 
extracted graphics data or ACTV data codes to the con- 
troller 260. The controller 260 interprets the extracted 
data as either control data, including instructions for 
switching between video signals, audio signals, or 
graphics data for on-screen display. If the data is on- 
screen display data, the data is preferably prefixed by a 
command designating the data as on-screen display 
data, as opposed to control data. Further, the controller 
260 also examines the control data for the occurrence of 
a header code designating the onset of a trigger point 
500 in the program, explained below. 
[0049] Figure 3 shows an alternate, dual tuner 
embodiment for seamless switching between separate 
video signals. This embodiment presumes that two 6 
MHz channels are used, each of which comprises com- 



pressed digital video and audio streams. In this embod- 
iment, the microprocessor 260 controls the selection of 
the RF channel that is demodulated by RF demodula- 
tors 200A, 200B. The demodulated data streams enter 
5 the forward error correctors 205A, 205B. At the output of 
the forward error correctors 205A, 205B the data 
streams are transmitted to the input of the digital demul- 
tiplexers 21 OA, 21 0B. 

[0050] As with the RF demodulators 200A, 200B, 

w the digital demultiplexers 21 OA, 21 0B are controlled by 
the microprocessor 260. This configuration allows the 
microprocessor 260 to independently select two differ- 
ent individual time-multiplexed video signals on different 
channels and data streams. If all the video signals of an 

is interactive program were contained on a single channel 
or data stream, it would only be necessary to have a sin- 
gle RF demodulator 200, forward error corrector 205, 
and digital demultiplexer 210 serially connected and 
feeding into the two digital video buffers 230A, 230B. 

20 [0051 ] Two data streams are provided from the dig- 
ital demultiplexers 21 OA, 21 0B. One data stream carries 
video information pertaining to the video signal the user 
is currently viewing. The second data stream carries the 
video signal selected based on the user's previous 

25 and/or current interactive selections from the user inter- 
face 270, as determined by the microprocessor 260. 
[0052] The digital information on each of the two 
streams is buffered in digital video buffers 230A, 230B. 
The buffered signals are then decompressed and con- 

30 verted into analog signals by decompressors/decoders 
21 5A, 21 5B which include digital to analog converters. 
The decompressors 21 5A, 21 5B are preferably MPEG- 
2 decoders. 

[0053] A local sync generator 225 is connected to 
35 sync add 220A, 220B and frame sync 380A, 380B cir- 
cuits. Because both streams are synchronized based 
on signals from the same local sync generator 225, 
each stream becomes synchronized to the other. In par- 
ticular, the signals on each stream are frame synchro- 
40 nized. 

[0054] A vertical blanking interval (VBI) switch 335 
is connected to the microprocessor 260 so that the input 
may be switched during the vertical blanking interval of 
the current stream, resulting in a seamless switch to the 
45 viewer. 

[0055] The embodiment of Figure 3 operates as fol- 
lows. Based on user responses and control codes, it is 
assumed that the microprocessor 260 determines that a 
switch from video signal A to video signal C should be 

so performed. The RF demodulator 200A and digital 
demultiplexer 21 OA are processing the currently viewed 
video signal, video signal A, which is progressing 
through the upper branch components. A command is 
issued from the microprocessor 260 to the RF demodu- 

55 lator 200A, 200B commanding a switch to the channel 
and data stream on which video signal C is located. The 
microprocessor 260 also instructs the digital demulti- 
plexer 21 0B to provide video signal C from the received 
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data stream to digital video buffer 230B. 
[0056] At this point, the upper RF demodulator 
200A and digital demultiplexer 21 OA are still independ- 
ently receiving and processing video signal A, which 
continues through the upper branch of the circuit. 
[0057] At a certain point, the digital decompres- 
sor/decoder 21 5B in the lower branch will begin filling 
up with video signal C frames. After video signal C is 
decompressed and decoded, it is converted into analog. 
A local sync generator 225 inserts both local sync and 
frame sync to video signal C via sync add circuit 220B 
and frame sync circuit 380B in order to synchronize it 
with the currently displayed video signal A, which is still 
being provided from the upper digital video buffer 230A. 
At the appropriate switch point, triggered by program- 
ming codes supplied with each video signal A and C, 
the microprocessor 260 directs the VBI switch 335 to 
switch in the vertical blanking interval from video A to 
video C, at which time video C will then seamlessly 
appear on the computer screen. 
[0058] Digital video buffers 230A, 230B may be 
used in the circuit of Figure 3, but are optional. However, 
in an alternative embodiment the buffers would be 
required to provide a seamless switch if the Figure 3 cir- 
cuit was modified to incorporate a single RF demodula- 
tor 200, single forward error corrector 205, and single 
digital demultiplexer 210, each with a single input and 
single output. In this alternative embodiment, the circuit 
cannot independently receive and demultiplex two data 
streams on different frequency channels. One buffer 
230A is used to store previously received video signals, 
while the other buffer 230B quickly passes through the 
selected video signals. 

[0059] Based on the same assumptions above, 
video signal A is progressing through the upper branch 
of the circuit and it is desired to switch to video signal C. 
However, in this alternative embodiment, the digital 
video buffer 230A is providing maximum buffering to 
video signal A. 

[0060] Because it is desired to switch to video sig- 
nal C, the microprocessor 260 directs the alternative cir- 
cuit (containing a single RF receiver 200, single forward 
error corrector 205 and single digital demultiplexer 210 
connected in serial), to receive and demultiplex the data 
stream on which video signal C is located, which may 
be different than that of video signal A. When video sig- 
nal C is demultiplexed, the microprocessor 260 directs 
the digital video buffer 230 to provide minimum buffering 
of video signal C so that decompressor/decoder 215 
may quickly decompress and decode the digital signals. 
After decompression and decoding, video signal C is 
synchronized with video signal A. At this time, video sig- 
nal A is read for display from digital video buffer 230A. 
The upper digital video buffer 230A must be large 
enough to provide video frames for output during the 
time it takes the RF demodulator 200 and digital demul- 
tiplexer 210 to switch to video signal C and the time 
required for decompression, decoding, and synchroni- 



zation of video signal C. 

[0061] When video signal C is synchronized with 
video signal A, the microprocessor 260 directs VBI 
switch 335 to switch from video signal A to video signal 

5 C in the vertical blanking interval of video signal A, 
thereby providing a seamless and flicker-free switch. 
[0062] At this time, digital video buffer 230 will begin 
to utilize maximum buffering by altering its fill/empty rate 
as described above with respect to the Figure 3 embod- 

w iment. When adequate buffering is achieved, a switch to 
another video signal may be performed in the same 
manner as described above. 

[0063] Another preferred embodiment is shown in 
Figure 4. This embodiment also includes an RF demod- 

15 ulator 200, a forward error corrector 205, and a digital 
demultiplexer 210. However, the circuitry differs along 
the rest of the chain to the television set or monitor. In 
this embodiment, a memory 475 is incorporated and 
connected to the output of the demultiplexer for storing 

20 the compressed composite digital video signal. The 
decompressor/decoder 215 is inserted at the output of 
the compressed memory. The decompressor/decoder 
21 5 decompresses the digital signal, converts the signal 
to analog and forwards the analog signal to the RF 

25 encode 245 for transmission to the monitor. Once the 
composite compressed digital video signal is fed into 
the compressed memory 475, the microprocessor 260 
directs a pointer to be placed somewhere along the 
compressed digital video signal. Based on the place- 

30 ment of the pointer, different frames and different seg- 
ments of the composite digital video signal will be read 
from memory 475 for decompression and decoding. 
[0064] The different video signals are distinguished 
from one another because they are labeled, preferably 

35 by headers. Assuming that video signal A has been 
selected for play on the monitor, the compressed digital 
memory 475 fills up with A frames. Assuming a switch to 
video signal C is desired, the microprocessor 260 
directs the RF demodulator 200 and digital demulti- 

40 plexer 210 to begin filling the compressed memory 475 
with video C frames. The decoder 21 5 pointer begins to 
move down. As soon as a sufficient number of C frames 
have entered the compressed memory 475, the pointer 
will then jump to the beginning of the C frames. The C 

45 frames are then output into the decompressor/decoder 
215 where the digital frames are converted into an ana- 
log signal. 

[0065] The digital video is multiplexed in a series of 
easily identifiable packets. These packets may contain 
so full compressed frames of video (I frames) or may 
include only the differences between full frames (B 
frames or P frames). 

[0066] To be able to reconstruct the full video 
images, the decompressor/decoder 215 needs to have 
55 a minimum number of I, P and B frames. The decoder 
215 needs only one I frame to decode an image. Con- 
versely, two prior Anchor frames (Ts M and "P's") are 
necessary to decode B frames. In order to decode P 
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frames, the decoder 215 only needs one Prior Anchor 
frame. When the microprocessor instructs the digital 
demultiplexer 210 to start sending packets from a differ- 
ent data stream there is no way to be certain that the 
next packet will be an I packet needed for decoding the 
second video stream. To avoid a breakup of the video 
images, which would occur if the decompres- 
sor/decoder 215 suddenly started receiving packets 
unrelated to the stream it was decoding, the microproc- 
essor 260 starts to fill up the memory 475 with video 
signal C packets until it is determined that a full 
sequence of I, B and P frames are available. The 
decoder 215 should receive the last bit of the last B 
frame in a given, GOP (Group of Pictures) before the 
switch, in order to prevent glitches when decoding. Fur- 
thermore, the last B frame of the GOP must only be 
backward predicted, not forward predicted or bidirec- 
tional predicted. As soon as the valid sequence is in 
memory 475 the microprocessor 260 moves the mem- 
ory read pointer to the start of a valid sequence of C 
video signal packets so that the decompressor/decoder 
215 can successfully decode the C signals. This results 
in a seamless switch from video signal A to video signal 
C. 

[0067] This embodiment requires a data channel for 
enabling a synchronous switch between a first video 
stream and a second video stream. This data channel 
comprises the ACTV codes which link together the dif- 
ferent program elements and information segments on 
the different video signals. In addition, the data channel 
also comprises synchronization pulses and a time code 
to signify to the pointer the proper time to skip from a 
memory location representing one video signal to a 
memory location representing another video signal in 
order to enable a seamless switch. 
[0068] The microprocessor 260 reads the data sig- 
nal from the digital demultiplexer 210 and communi- 
cates pertinent data to the sync add circuit 220, which is 
connected to sync generator 225. The microprocessor 
260 is then able to synchronously communicate with the 
memory 475. 

[0069] The time code sent will identify the timing for 
one picture, as well as for multiple pictures, and will lock 
the different pictures together. This is done through the 
use of similar clocks at both the transmission end and 
the receiver. A time code is used in order to keep the 
two clocks at both the transmission and receive end 
synchronously connected to one another. Once the 
clocks at both ends are working synchronously, each of 
the multiplexed video streams must be synchronized to 
the clocks. In order to synchronize the multiplexed video 
stream to the clocks, each of the individual channels 
must be referenced to a common reference point and 
must be identified. 

[0070] In the preferred embodiment, a packet 
header would be incorporated into the transport layer of 
the MPEG signal to identify the various channels. The 
packet header will also include information as to where 



to insert the vertical blanking interval. In MPEG, the ver- 
tical blanking interval is not transmitted from the head- 
end. Therefore, the vertical blanking interval must be 
generated locally. The packet header eye will identify at 
5 what time the vertical blanking interval is in existence in 
order to effectuate a seamless switch between analog 
pictures. 

[0071] In summary, the combination of clock and 
the information embedded in either the transport layer 

10 of MPEG or in a separate packet on a separate data 
channel effectuates the linking between each video sig- 
nal and a corresponding time point. The data channel 
also includes information designating when all the vari- 
ous video signals will be in synchronism with one 

15 another. It is at these points that the microprocessor 260 
may direct the pointer to skip from one location to 
another location, at a time (such as during the VBI) 
when a seamless switch will result. 

20 D. Trigger Points 

[0072] Interactivity is further enhanced in the digital 
interactive embodiments through the application of trig- 
ger points 500 scattered at various predetermined times 

25 throughout the program, a timeline representation of 
which is shown in Figure 5. The trigger points 500 cor- 
respond to times when interactive events are scheduled 
to take place during the live sporting event 10. These 
interactive events could be the selection and playing of 

30 video, audio segments, the display of graphics or dis- 
play of Web pages accessed from Internet Web sites 
170. For example, when a viewer's favorite baseball 
player is at bat, graphics showing past or current per- 
formance statistics of the player can be overlaid on the 

35 screen while excerpts from an interview with the player 
can be played for the viewer. While the choice of partic- 
ular video, audio or graphics is still dependent on viewer 
selections, the viewer selections in response to dis- 
played graphical interrogatory messages are preferably 

40 made during a period at the onset of the program or 
when a viewer first tunes into the program. Alternatively, 
interrogatories are not necessary if the switches are 
based on the viewer profile stored in memory 265. 
These viewer selections are then utilized as inputs to 

45 macros called up at later times during the program by 
the controller 260 upon the occurrence of the trigger 
points 500, identified to the interactive computer by 
unique codes embedded in the video signal. 
[0073] The trigger points 500 correspond to the 

50 times when the conventional program content can be 
altered and personalized for the viewers. The program- 
mer can place the trigger points 500 at any time 
throughout the program. Since the trigger points 500 
are unknown to the subscriber, the subscriber does not 

55 know when they will receive a personalized message. In 
other words, an interactive response can either immedi- 
ately follow a corresponding user selection made to an 
interrogatory message or occur at a later time corre- 
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sponding to a trigger point 500, or any combination of 
the two. Of course, timing of the interactive events 
should correspond to suitable times in the program 
where branching to interactive elements is sensible and 
does not clash with the program content of the conven- 
tional video still displayed on the television 165 or other 
display monitor. 

[0074] At the onset of a trigger point 500, the con- 
troller 260 will select one of several possible audio (or 
video or graphic display) responses for presentation to 
the subscriber. As mentioned above and shown in figure 
5, some of the responses may comprise a branch to 
either a video segment, graphics and/or audio seg- 
ments. 

[0075] In combination with the use of trigger points 
500, the present invention allows for the viewer to select 
certain options at the onset of the program to suit the 
viewers' preferences. For example, if the program 
broadcast is a live sports event 10, at an early trigger 
point 500, the viewer could be queried as to whether the 
viewer would prefer to receive audio in English, Span- 
ish, French, or perhaps hear the local announcer 
instead of the network announcer. Upon the viewer 
selection, the CPU 260 directs a branch to the appropri- 
ate interactive segment. 

[0076] Each trigger point 500 is identified preferably 
through the broadcast of ACTV codes sent as part of 
the composite interactive program signal. The codes 
preferably include, at a minimum, the following informa- 
tion: (1) header identifying the occurrence of a trigger 
point 500; (2) function ID (e.g., selection of audio or 
graphics responses, etc.); and (3) corresponding inter- 
rogatory message(s) or particular viewer characteristic 
or habit based on viewer profile. The first bit sequence 
simply identifies to the controller that a trigger point 500 
is about to occur. The function ID designates the macro 
or other set of executable instructions for the controller 
260 to read and interpret to obtain the desired result, 
e.g., a selected video and/or audio response. 
[0077] Upon extraction of the codes by the data 
decoder, the controller 260 reads and interprets the 
codes and calls from memory 265 a particular user 
selection(s) designated by the trigger point 500 codes. 
The user selections correspond to subscriber answers 
to a series of interrogatory messages preferably pre- 
sented at the beginning of the program. After obtaining 
the appropriate user selection(s), the controller 260 
reads and performs the executable instructions using 
the user selection(s) as input(s) in the macro algorithm. 
The result of the algorithm is either a selected video 
stream, audio and/or selected graphics response. The 
video/audio response can be called from memory 265 if 
it is prestored, called from external data storage, or the 
controller 260 can command the switch to branch to the 
particular video audio stream if the response is broad- 
cast concurrently with the trigger point 500. After the 
selected video/audio response is played to the sub- 
scriber, the switch branches back to the standard pro- 



gram, shown at time t s in Figure 5. 
[0078] As mentioned above, a series of interroga- 
tory messages are preferably presented when the sub- 
scriber begins watching the interactive program. These 

5 interrogatory messages can be presented in any one of 
three ways. First, the interrogatory messages can be 
presented as graphics displays overlaid by the interac- 
tive computer workstation onto a video signal, wherein 
the graphics data is sent in the vertical blanking interval 

10 of the composite interactive signal, or alternatively 
stored on the hard disk or external storage. Second, the 
interrogatory messages are presented as graphics dis- 
plays as discussed above, except the graphics data 
comes from local storage, external data storage (e.g., 

15 CD ROM, cartridge, etc.), or a combination of data in 
the VBI and data called from eKher local or external data 
storage. Third, graphics data can be presented in the 
form of user templates stored at the interactive compu- 
ter workstation. 

20 [0079] User selections corresponding to answers to 
the n successive interrogatory messages are received 
by the remote interface 270 at the beginning of the 
show, stored in memory 265 and used throughout the 
show at the appropriate trigger points 500 to subtlety 

25 change program content as the show progresses. Pref- 
erably, each interrogatory has a set of possible answers. 
Next to each possible answer will be some identifier cor- 
responding to a label on a key on the user interface. The 
subscriber depresses the key corresponding to their 

30 answer selection. This selection is decoded by the 
remote interface 270 and controller 260, stored in mem- 
ory 265, preferably RAM, and used later as required by 
an algorithm designated at a trigger point 500. 

35 E. internet 

[0080] In addition to the central studio serving as a 
source of interactive option responses, the Internet can 
be used as a source of personalized information for 

40 interactive responses. 

[0081 ] As discussed above, the video programming 
is preferably created at a centralized location, i.e., the 
control studio 5 as shown in Figure 1 , for distribution to 
subscribers in their homes. Referring to Figures 1 and 

45 6, in a preferred method, the operator at the control stu- 
dio 5 must designate certain Web pages to correspond 
to one of the program options, such as audio and/or 
graphics options, using control PC 135 and URL 
encoder 600. Instead of encoding the actual content of 

so the options at the control studio 5, as with the audio, 
Web address identifiers, i.e., Uniform Resource Loca- 
tors (URLs) are encoded and sent as part of the data 
codes from the control PC 135 to the digital multiplexer 
140. After multiplexing, the URLs are sent as part of the 

55 program signal 61 5, as described above. Preferably, the 
URLs, like the various audio and graphics options, have 
associated time stamps which indicate to the remote 
digital set top boxes 25 when, during the video program, 
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to display the particular Web pages addressed by the 
URLs, the selection and display of which is preferably 
made as a function of viewer responses or viewer pro- 
file. 

[0082] Preferably, each digital set top box 25 has an 
Internet connection 160 created concurrently with the 
cable connection. The Internet connection 160 can be 
via high-speed line, RF, conventional modem. The dig- 
ital set top box 25 has Internet access 1 60 via any of the 
current ASCII software mechanisms. In a preferred 
embodiment, in the interactive digital set top box 25, the 
digital demultiplexer 210 extracts the URLs along with 
the other data codes. In an alternative embodiment, a 
local URL decoder 605 at the user site extracts the 
URLs. 

[0083] In a preferred embodiment, a JAVA enabled 
browser as well as specialized software for performing 
part of the method of the present invention are installed 
on the interactive digital set top box 25. The JAVA ena- 
bled browser allows the interactive digital set top box 25 
to retrieve the Web pages and is preferred software, 
since it is platform independent, and thus, enables effi- 
cient and flexible transfer of programs, images, etc., 
over the Internet. The specialized software acts as an 
interface between the video programming and the Inter- 
net functions of the present invention. The processor 
and software interprets these URLs and directs the 
JAVA enabled browser to retrieve the particular relevant 
Web pages, and synchronizes the retrieved Web pages 
to the video content for display on the television monitor 
165 at the appropriate times. 

[0084] In the present invention, the viewer also has 
the capability to link to a channel website at will. For 
example, if a viewer is interested in purchasing a prod- 
uct described in an advertisement, by merely clicking on 
a button on their remote 20, the producer's Website 
could be accessed by Internet connection 160 and dis- 
played to the viewer. The viewer could then either obtain 
more information about the product or order the prod- 
uct, if desired. As described above, this application is 
possible by sending the URL associated with the pro- 
ducer's Website to the digital cable boxes 25 as part of 
the interactive program. Upon selection by the viewer, 
the web browser, located either in the digital set-top box 
25 or externally in a connected PC 610, can retrieve the 
Web pages. The specialized software then synchro- 
nizes the Web pages for video display. 

F. Memory 

[0085] The interactive digital set top box 25 of the 
present invention also has the advantage of remember- 
ing subscriber responses and using these responses in 
choosing a video/audio response, and/or graphics inter- 
rogatory message, to present to the student. Memory 
branching is a technique of the present invention where 
the algorithm assembles video/audio responses and 
graphics interrogatory messages according to the cur- 



rent and previous user inputs. Memory branching is 
accomplished by linking video/audio streams and/or 
successive graphics interrogatory messages together in 
a logical relationship. In this scheme, the interactive dig- 

5 ital set top box 25 contains logic (preferably, in the soft- 
ware algorithm) and memory 265 to store previous 
subscriber selections and to process these previous 
responses in the algorithm to control future video/audio 
stream selection, as well as future graphics message 

10 selection. 

G. Dig it al Viewer Profiles 

[0086] In a preferred embodiment, the interactive 

15 digital cable box 25 can have a "viewer profile" stored in 
its memory 265. Alternatively, the accumulated profile of 
viewer characteristics and/or habits can be stored at the 
control studio 5 or cable headend. If the profile statistics 
are accumulated at some central location, they can also 

20 be sent to each viewer's home. On the other hand, if 
accumulated in memory 265 at each of the set top ter- 
minals 25, the data could be sent to the central location 
for storage and dissemination by way of the digital back 
channel. Decisions regarding personalized advertising 

25 and viewing can then be made for a viewer or class of 
viewers based on the accumulated parameters. 
[0087] The "viewer profile" preferably contains char- 
acteristics of the particular viewer at that subscriber 
location, such as sex, hobbies, interests, etc. This 

30 viewer profile is created by having the viewer respond to 
a series of questions. Alternatively, the viewer profiles 
could be created at a control studio 5 and sent to the 
interactive digital cable box 25. This information is then 
used by the cable box software to create a compendium 

35 of the viewer's interests and preferences -- i.e., a user 
profile. The stored user profile would be used in place of 
the question/answer format, and thus, dictate the 
branches to interactive segments of interest to the 
viewer. 

40 [0088] Alternatively, the interactive cable box 25 
can be programmed to create a user profile of each 
viewer based on the selections made during one of the 
interactive programs. If the profile shows that a particu- 
lar viewer does not enjoy violence, the system can auto- 

45 matically branch to another video signal at the 
commencement of a fight in a sports program. Further- 
more, such a user profile could be modified or enriched 
over time based on selections made during future inter- 
active programs. For example, the 'memory' technique 

so described above can be used to modify the user profile 
based on user response over time. 
[0089] Event data is collected from the viewer when 
the following command is received and processed by 
the controller 260 in the interactive digital cable box 25: 

55 

VIEWER PROFILE 

This command is used to enable events which 
can be utilized for profiling during a show. The 
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events indicated in this command are those profil- 
ing events which are possible during the show. The 
enabled events may be either selected by the 
viewer during the show, or may be automatically 
enabled based upon viewer selections. 5 

VIEWER.PRORLE EVENT1,...EVENT24 

[0090] Once the profile is created, the programming 
choices or interactive responses can be triggered based 
on the content of the viewer profile itself. For example, if 
the viewer profile suggests that the viewer is particularly 
interested in sports cars, a sports car commercial could 
be played for the viewer at a predetermined point in the 
program. As another application, if a viewer's profile 
indicates that the viewer is interested in cooking, when- 
ever the viewer watches such a program, the user pro- 
file would trigger the interactive program to download 
recipes and either display such recipes on the screen or 
send the recipes to an attached printer. 
[0091] Viewer profile information can then be col- 
lected at the control studio 5 through polling of the 
viewer cable boxes 25 for viewer selection data. Special 
polling software is loaded into the digital set top box 25 
for performing the polling functions. Alternatively, the 
invention allows for the digital set top boxes 25 to send 
back data on command from the control studio 5 or peri- 
odically. The command to initiate an upload of viewer 
profile data is as follows: 

UPLOAD EXTENDED 

This command is used to initiate the uploading 
of data to the central site. 

UPLOADJEXTENDED UPLOADJDENTIFIER 
VARIABLE_NAME [PHONE_NUMBER] 

UPLOADJDENTIFIERnumeric constant identifying 
upload. 

VARIABLE_NAME name of variable to be 
uploaded 

PHONE_NUMBER string, that consists of num- 
bers only, it can be omitted, if 
not needed. 

Regardless of whether a polling or periodic scheme is 
used, the statistics and other user profile information is 
preferably sent back to the control studio 5 by use of the 
back-channel. 

H. Applications 

[0092] The embodiments, described above, allow 
for several possible applications. For example, in a live 
sports event 10, one channel could carry the standard 
video channel, with other channels carrying different 
camera angles 100 and/or close-ups of particular play- 
ers. Other potential video options include instant replay, 



highlights, player statistics via graphic overlays, etc. 
Graphics presenting statistical information on the play- 
ers can be constantly updated using the Chyron sys- 
tem. 

[0093] The provision of trigger points 500, 
explained above, can provide for seamless integration 
of such video options during the sporting event, based 
on either the viewer responses to interrogatories at the 
beginning of the program and/or on the digital viewer 
profile. 

[0094] Further, the viewer can become the director 
with the present invention. For example, the viewer can 
choose which camera angles 100 to emphasize. In a 
broadcast of golf, the viewer can direct whether they 
desire to follow a particular player from hole-to-hole, 
focus on one particularly difficult hole. In this manner, 
the viewer can customize the sporting broadcast to 
meet his own interests. 

[0095] Further, the viewer can act as a director to 
create their own video. During a live concert 10, for 
example, the viewer selects various camera angles 100 
at different times. These selections along with a time 
stamp, indicating the program time when each selection 
was made, are stored in memory 265. When the pro- 
gram is played back a second time, the processor 260 
will automatically direct branching between the video 
channels according to the stored selections at the time 
stamp. In this manner, a music video can be created by 
the viewer. 

[0096] The viewing experience can be further 
enlightening for the viewer by implementing games and 
contests during the live sporting event 10. For example, 
graphics overlays can be developed that query the 
viewer during the game. During a football broadcast, for 
example, viewers can be queried with such interrogato- 
ries as the following: 

What will be the next play? (RUN/PASS/KICK); 

Will the offense get the first down?; 

Will they score on this possession?; 

Pick the halftime score; 

Who will win? 

[0097] Each viewer's responses can be sent back 
to the control studio 5 for tabulation of scores. Prefera- 
bly, the responses are packaged at the digital cable box 
25 and transmitted to the control studio 5 via the digital 
backchannel upon the UPLOAD EXTENDED com- 
mand. Alternatively, tabulation of scores can take place 
at the digital cable box 25 through the utilization of cer- 
tain software in memory 265. Each correct answer can 
correspond to a certain number of points. At the end of 
the game, the interactive program preferably presents a 
graphic showing the viewer point total. If desired, adver- 
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tisers could present special gift certificates for excellent 
performance in such games. The provision of such cer- 
tificates would occur by displaying a certain code that a 
viewer can take to a store to receive the gift. In this man- 
ner, viewer interests in sports events can be enhanced. 5 
[0098] Further, the viewer has the option with the 
present invention to block out viewing of certain events. 
For example, if the viewer is adverse to violence during 
a sporting event or other type program, the system can 
block out such options from the viewer with the following 10 
data commands: 

EVENT 

This command is used to indicate occurrence 
of a certain event (e.g., a fight breaking out during a 15 
football game). 

EVENT EVENT.NUMBER 

EVENT_NUMBER is a numeric constant. 

20 

ON EVENT EXECUTE MACRO 

This command is used to implement certain 
actions (such as an automatic branch to another 
video channel in order to block out violent event, for 
example), as soon as the event arrives. 25 

ON_ EVENT EVENT, NUMBER MACRO. 
NUMBER 

EVENT_NUMBER is a numeric constant. 30 
MAC RO_N UMBER is a numeric constant. 
I. Two-Way Configuration 

35 

[0099] The live programming system of the present 
invention may be operated in a two way configuration, 
as illustrated in Figure 7. In this mode, the various video 
signals are processed as previously described, being 
digitized and compressed at the control studio 5. The 40 
signals are then sent to a central switching station, or 
headend 30. 

[0100] In this embodiment, the switching between 
the various live digital signals is accomplished at the 
headend 30 rather than at the receiver. On the receive 45 
end, each digital set-top box 760 relays viewer selec- 
tions back to the remotely located switching station 30. 
Preferably, the viewer selections are relayed by way of 
the digital back channel 770. However, the viewer selec- 
tions may be relayed to the switching station 30 by any so 
conventional means, such as two-way cable television, 
telephone or microwave transmission. The switching 
station 30 receives the viewer selection and routes the 
desired signal to a transmitter 750 which conventionally 
transmits the desired video down the appropriate digital 55 
cable channel for the particular viewer. 
[0101] At the central switching station 30, a demul- 
tiplexer 710 demultiplexes the compressed signals and 



places each on a separate bus channel 725. A number 
of remote control interactive switches 730, 732, 734, 
736 are connected to the video signal bus 725. Based 
on the viewer selections, an algorithm stored in memory 
265 and under processor 260 control at the central 
switching station 30, a digital seamless switch is made 
and the selected video, audio and/or graphics are for- 
warded to the viewer home for display. 
[01 02] Such a two-way embodiment could be imple- 
mented in a video dial tone or video server system. In 
such a system, only a single video channel 755 is nec- 
essary for each home. Once the viewer selection is 
received at the server site at the cable headend 30, a 
switch is made to the appropriate video stream and this 
stream is sent on the single channel 755 to the home. 
[0103] Alternatively, it may be desirable to transmit 
an interactive sporting event over a single telephone 
line. When the viewer enters a selection on their remote 
20, a signal is sent by way of the telephone line to the 
central switching station 30 which routes the desired 
signal of the interactive program over the user's tele- 
phone line so that a single link handles both the interac- 
tive choice being made at the receiver and the 
transmission of that choice from the headend 30 where 
the actual switching takes place in response to the inter- 
active selection made at the receiver. 
[0104] The two-way link between the viewer and 
switching station 30 may be used for other purposes. 
For example, demographic data may be transferred 
from the viewer to the broadcast network for commercial 
purposes, such as targeted advertising, billing, or other 
commercial or non-commercial purposes. 
[0105] While the present invention has been 
described primarily with respect to live events, and in 
particular sporting events, it has equal potential for 
enhancing content in other program categories. A 
viewer can become their own director of a murder mys- 
tery or other drama. By entering responses to displayed 
questions at the initiation of or during the show, the pro- 
gram will branch to alternative video/audio segments as 
a result of the user selections. In this manner, different 
viewers with different selections may end tip with a dif- 
ferent murderer at the conclusion of the broadcast. 
[0106] Using the foregoing embodiments, methods 
and processes, the interactive multimedia computer 
maximizes personalized attention and interactivity to 
subscribers in their homes in real time. Although the 
present invention has been described in detail with 
respect to certain embodiments and examples, varia- 
tions and modifications exist which are within the scope 
of the present invention as defined in the following 
claims. 

Claims 

1 . A method of providing to a user digital programming 
at a receiver station, comprising the steps of: 
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receiving a plurality of digitally compressed 
video signals, each signal corresponding to a 
different video option of a program; 

creating a profile for the user; 5 

selecting one of the video options, wherein the 
video option is selected based upon the user's 
profile resulting in a program tailored to the 
user; 10 

digitally decompressing the selected video sig- 
nal corresponding to the selected video option; 
and 

15 

displaying the selected video signal corre- 
sponding to the selected video option, wherein 
visual transition to the selected video signal is 
seamless. 

20 

2. A method as claimed in Claim 1 , further comprising 
the step of storing the user profile in memory at the 
receiver station. 

3. A method as claimed in Claim 1 , wherein the user 25 
profile is stored in memory at a central location. 

4. A method as claimed in any of Claims 1 to 3, 
wherein the user profile contains data indicating the 
user's viewing characteristics. 30 

5. A method as claimed in any preceding claim, 
wherein the user profile contains data indicating a 
user's personal profile. 

35 

6. A method as claimed in any preceding claim, 
wherein at least one of the video signals is an 
advertisement. 

7. A method as claimed in any preceding claim, ao 
wherein the user profile contains selections made . 
by the user during an interactive program. 

8. A method as claimed in any preceding claim, 
wherein the user profile contains data collected 45 
from user responses to interrogatories, wherein the 
interrogatories are part of the program. 

9. A method as claimed in any preceding claim, 
wherein data for the user profile is collected by poll- so 
ing the receiver station from the central location. 

10. A method as claimed in any preceding claim, 
wherein the receiver station is a digital television. 

55 

11. A method as claimed in any of Claims 1 to 9, 
wherein the receiver station comprises a personal 
computer with a television card. 



12. A method as claimed in any of Claims 1 to 9, 
wherein the receiver station comprises a digital 
cable box and a television, operably connected to 
the digital cable box. 

13. A method as claimed in any preceding claim, fur- 
ther comprising the steps of: 

indicating to the user the different video 
options; 

receiving from the user a command indicating 
the selected video option. 

14. A method as claimed in any preceding claim, 
wherein the plurality of video signals further com- 
prise at least one closeup video and at least one 
stow motion video replay. 

15. A method as claimed in any preceding claim, 
wherein the selected video option corresponds to 
the replay video. 

16. A method as claimed in any preceding claim, 
wherein the digital programming is received from a 
digital versatile disk. 

17. A method as claimed in any of Claims 1 to 15, 
wherein the digital programming is received from a 
cable headend. 

18. A method as claimed in any of Claims 1 to 15, 
wherein the digital programming is received from 
satellite broadcast. 

19. A method as claimed in any preceding claim, fur- 
ther comprising the steps of: 

obtaining a plurality of graphics segments; 

selecting at least one graphic segment; 

displaying the selected graphic segments. 

20. A method as claimed in Claim 19, wherein at least 
one of the graphics segments is stored locally to the 
receiver station. 

21. A method as claimed in Claim 19, wherein at least 
one of the graphics segments is received over a 
communications link from a remote source. 

22. A method as claimed in Claim 19, wherein at least 
one of the graphics segments is received over a 
communications link with an online service pro- 
vider. 

23. A method as claimed in any preceding claim, fur- 
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ther comprising the step of receiving a plurality of 
audio signals. 

24. A method as claimed in Claim 23, wherein each 
audio signal is associated with one of the video sig- 
nals. 



wherein at least one of the video signals is an 
advertisement. 

34. A system as claimed in any of Claims 28 to 33, 
5 wherein the user profile contains selections made 
by the user during an interactive program. 



25. A method as claimed in Claim 23 or Claim 24, 
wherein at least one of the audio signals is stored 
locally to the receiver station. 

26. A method as claimed in Claim 23 or 24, wherein at 
least one of the audio signals is received over a 
communications link from a remote source. 

27. A method as claimed in Claim 23 or Claim 24, 
wherein at least one of the audio signals is received 
over a communications link with an online service 
provider. 

28. A system of providing to a user digital programming 
at a receiver station, comprising: 

means for receiving a plurality of digitally com- 
pressed video signals, each signal correspond- 
ing to a different video option of a program; 

a processor, operably connected to the receiv- 
ing means, for selecting one of the video 
options, wherein the video option is selected 
based upon a user profile resulting in a pro- 
gram tailored to the user; 

a means for digitally decompressing the 
selected video signal corresponding to the 
selected video option; and 

a means for displaying the selected video sig- 
nal corresponding to the selected video option, 
wherein visual transition to the selected video 
signal is seamless. 

29. A system as claimed in Claim 28, wherein the user 
profile contains data indicating the user's viewing 
characteristics. 



35. A system as claimed in any of Claims 28 to 34, 
wherein the user profile contains data collected 

10 from user responses to interrogatories, wherein the 
interrogatories are part of the program. 

36. A system as claimed in any of Claims 28 to 35, 
wherein data for the user profile is collected by poll- 

15 ing the receiver station from a central location. 

37. A system as claimed in any of Claims 28 to 36, 
wherein the receiver station is a digital television. 

20 38. A system as claimed in any of Claims 28 to 36, 
wherein the receiver station comprises a personal 
computer with a television card. 

39. A system as claimed in any of Claims 28 to 36, 
25 wherein the receiver station comprises a digital 

cable box and a television, operably connected to 
the digital cable box. 

40. A system as claimed in any of Claims 28 to 39, 
30 wherein the digital programming is received from a 

digital versatile disk. 

41. A system as claimed in any of Claims 28 to 39, 
wherein the digital programming is received from a 

35 cable headend. 

42. A system as claimed in any of Claims 28 to 39, 
wherein the digital programming is received from 
satellite broadcast. 



40 

43. A system as claimed in any of Claims 28 to 42, 
wherein the plurality of video signals represent 
advertisements. 

45 



30. A system as claimed in Claim 28 or Claim 29, fur- 
ther comprising a means for storing the user profile. 

31. A system as claimed in any of Claims 28 to 30, so 
wherein the user profile contains data indicating a 
users personal profile. 

32. A system as claimed in any of Claims 28 to 31 , 
wherein the user profile storage means is located at 55 
a central location. 

33. A system as claimed in any of Claims 28 to 32, 
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